Contrastive learning (CL) explicitly enlarges the feature representation similarity between semantic-relevant samples, and it is adept at capturing high-level semantics while discarding irrelevant information.

In semi-supervised learning (SSL), a common practice is to learn consistent information from unlabeled data and discriminative information from labeled data to ensure both the immutability and the separability of the classification model. Existing SSL methods suffer from failures in barely-supervised learning (BSL), where only one or two labels per class are available, as the insufficient labels cause the discriminative information being difficult or even infeasible to learn. To bridge this gap, we investigate a simple yet effective way to leverage unlabeled samples for discriminative learning, and propose a novel discriminative information learning module to benefit model training. Specifically, we formulate the learning objective of discriminative information at the super-class level and dynamically assign different classes into different super-classes based on model performance improvement. On top of this on-the-fly process, we further propose a distribution-based loss to learn discriminative information by utilizing the similarity relationship between samples and super-classes. It encourages the unlabeled samples to stay closer to the distribution of their corresponding super-class than those of others. Such a constraint is softer than the direct assignment of pseudo labels, while the latter could be very noisy in BSL. We compare our method with state-of-the-art SSL and BSL methods through extensive experiments on standard SSL benchmarks. Our method can achieve superior results, \eg, an average accuracy of 76.76\% on CIFAR-10 with merely 1 label per class.

Multi-view clustering is an important research topic due to its capability to utilize complementary information from multiple views. However, there are few methods to consider the negative impact caused by certain views with unclear clustering structures, resulting in poor multi-view clustering performance. To address this drawback, we propose self-supervised discriminative feature learning for deep multi-view clustering (SDMVC). Concretely, deep autoencoders are applied to learn embedded features for each view independently. To leverage the multi-view complementary information, we concatenate all views' embedded features to form the global features, which can overcome the negative impact of some views' unclear clustering structures. In a self-supervised manner, pseudo-labels are obtained to build a unified target distribution to perform multi-view discriminative feature learning. During this process, global discriminative information can be mined to supervise all views to learn more discriminative features, which in turn are used to update the target distribution. Besides, this unified target distribution can make SDMVC learn consistent cluster assignments, which accomplishes the clustering consistency of multiple views while preserving their features' diversity. Experiments on various types of multi-view datasets show that SDMVC outperforms 14 competitors including classic and state-of-the-art methods. The code is available at https://github.com/SubmissionsIn/SDMVC.

M) null /τ null, (10) which is derived by considering Equation 4. To simplify the equation, we hold L M is the dimensional mask. Note that the scalar derivation, e.g., the MetaMask's training paradigm as follows. In order to prove Theorem 5.2 and the conclusion that the bounds of supervised cross-entropy loss A.2.1 Proof for the Equality Part To prove Φ null g To prove Equation 20, we demonstrate an evidence example in Figure 5. The reason behind such a phenomenon is that, following Theorem 5.1, the self-paced dimensional mask jointly enhances the gradient Being aware of proofs in Section A.2.1 and Section A.2.2, we confirm the validation of Theorem Then, we bring Theorem 5.2 into Theorem 5.1 to derive the comparison of the lower bounds of Therefore, the lower bound obtained by the masked representation, i.e., MetaMask, is larger than the Concretely, we conclude that our approach can better bound the downstream classification risk, i.e., However, our dimensional confounder is defined as a negative factor that may lead to model degradation, which is proposed from the dimensional perspective. MetaMask using a trick of fixed learning rate instead of the cosine annealing strategy.

Our privacy computation is based on the notion of Rényi-DP, which we recall as follows. Lastly, we use the following theorem to convert ( α,ε) -RDP to (ε, δ) -DP . The total dataset size is 60K for the training set and 10K for the testing set, respectively. All our models and methods are implemented in PyTorch. Based on Google's TensorFlow privacy under version The experiments presented in Section 5.2 of the main paper correspond to the class-incremental learning setting [ And the task protocol is sequentially learning to classify a given sample into all the classes seen so far.

'SVD-Large' indicates that we use the eigenvec-tors corresponding to larger singular values to build the generalization space in which localization is

Contrastive learning (CL) explicitly enlarges the feature representation similarity between semantic-relevant samples, and it is adept at capturing high-level semantics while discarding irrelevant information.